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METHODS, DEVICES, AND COMPUTER PROGRAM PRODUCTS FOR 
CHANGING TIME INTERVALS OF THE OCCURRENCE OF AUDIO 
INFORMATION FROM LOCAL AND REMOTE SITES 

FIELD OF THE INVENTION 
The invention generally relates to the field of information processing and, 
more particularly, to methods, systems, and computer program products for 
processing audio information. 

BACKGROUND 

It is known that momentary periods of silence exist during normal 
conversations. For example, in a question and answer session, there is usually a 
period of silence that occurs between the end of a question the answer provided 
thereto. Furthermore, periods of silence can also occur during normal conversation 
between two or more parties. Accordingly, this momentary silence is more or less 
expected is normal situations. 

It is also known that in some situations (which fall outside the realm of a 
normal conversation) unusually long periods of silence can be introduced where 
otherwise normal periods of silence would occur. For example, when a conversation 
is occurring over a great distance, such as via a satellite link, the propagation delay 
associated with the transmission of audio from a local site to a remote site coupled 
with the propagation of audio information from the remote site back to the local site 
can introduce such long periods of silence. These long periods of silence during the 
conversation can become awkward and unproductive as the participants may be 
unsure as to how long to wait before speaking again. 

It is known to address the situation discussed above by recording an entire 
conversation in a medium which can allow "off-line" editing to reduce the amount of 
silence present in the conversation. For example, a question and answer session 
conducted by via satellite can be recorded on audiotape whereupon the recording can 
be edited to reduce any silence between the questions and answers. However, this 
approach may not be feasible when the session is to be carried out in real-time or near 
real-time. 
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SUMMARY 

Embodiments according to the invention can provide methods, devices, and 
computer program products for changing a time interval in which audio information 
occurs to provide time-changed audio information that can be combined with other 
5 audio information for broadcast. For example, in some embodiments according to the 
invention, local audio information can be broadcast over a satellite link to a remote 
site. The local audio information can be time-expanded so that the time interval 
during which the time-expanded audio occurs occupies some of the interval which 
would otherwise be silent due to the propagation delay of response from the remote 

10 site. When the remote audio information is received at the local site, the remote audio 
information can be combined with the time-expanded audio to provide audio 
information for broadcast to an audience. Therefore, embodiments according to the 
invention may allow an awkward silence that would otherwise exist in an interval 
between the end of the local audio and the beginning of the remote audio to be 

15 reduced to a comfortable level. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1A is a block diagram that illustrates embodiments of methods, 
systems, and computer program products according to the invention. 
20 Figure IB is a schematic timing diagram that represents propagation delay 

associated with transmissions between local and remote sites. 

Figure 1C is a schematic timing diagram that illustrates broadcast audio that 
can be provided by methods, systems, and computer program products according to 
the invention. 

25 Figure 2 is a flowchart that illustrates operations of embodiments of methods, 

systems, and computer program products according to the invention. 

Figure 3A is a block diagram that illustrates embodiments of methods, 
systems, and computer program products according to the invention. 

Figure 3B is a schematic timing diagram that illustrates situations where audio 
30 from local and remote sites occur in an overlapping time interval. 

Figure 3C is a schematic timing diagram that illustrates broadcast audio 
information that includes time-expanded and time-compressed audio information 
according to the invention. 
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Figure 4 is a flowchart that illustrates operations of embodiments of methods, 
systems, and computer program products according to the invention. 

Figure 5 is a schematic representation of a network over which 
communications can occur according to the invention. 

5 

DESCRIPTION OF EMBODIMENTS ACCORDING TO THE INVENTION 
The invention is described herein with reference to the accompanying 
drawings, in which embodiments of the invention are shown. This invention may, 
however, be embodied in many different forms and should not be construed as limited 

10 to the embodiments set forth herein; rather, these embodiments are provided so that 
this disclosure will be thorough and complete, and will fully convey the scope of the 
invention to those skilled in the art. Like numbers and reference designators refer to 
like elements throughout. 

As will be appreciated by one of skill in the art, the present invention may be 

15 embodied as methods, systems, and/or computer program products. Accordingly, the 
present invention may take the form of an entirely hardware embodiment, an entirely 
software embodiment or an embodiment combining software and hardware aspects. 
Furthermore, the present invention may take the form of a computer program product 
on a computer-usable storage medium having computer-usable program code 

20 embodied in the medium. Any suitable computer readable medium may be utilized 
including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices. 

Computer program code or "code" for carrying out operations according to the 
present invention may be written in an object oriented programming language such as 
JAVA®, or in various other programming languages. Software embodiments of the 

25 present invention do not depend on implementation with a particular programming 
language. Portions of the code may execute entirely on one or more systems utilized 
by an intermediary server. 

The code may execute entirely on one or more servers, or it may execute 
partly on a server and partly on a client within a client device or as a proxy server at 

30 an intermediate point in a communications network. In the latter scenario, the client 
device may be connected to a server over a LAN or a WAN (e.g., an intranet), or the 
connection may be made through the Internet (e.g., via an Internet Service Provider). 
It is understood that the present invention is not TCP/EP-specific or Internet-specific. 
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The present invention may be embodied using various protocols over various types of 
computer networks. 

The invention is described below with reference to diagram illustrations of 
methods, systems and computer program products according to embodiments of the 
5 invention. It is understood that each block shown in the diagrams (i.e., block diagram 
and/or flowchart diagram), and combinations of blocks in the diagrams, can be 
implemented by computer program instructions. These computer program 
instructions may be provided to a processor of a general purpose computer, special 
purpose computer, or other programmable data processing apparatus to produce a 

10 machine, such that the instructions, which execute via the processor of the computer 
or other programmable data processing apparatus, create means for implementing the 
functions specified in the diagram block or blocks. 

These computer program instructions may be stored in a computer-readable 
memory that can direct a computer or other programmable data processing apparatus 

15 to function in a particular manner, such that the instructions stored in the 

computer-readable memory produce an article of manufacture including instruction 
means which implement the function specified in the diagram block or blocks. 

The computer program instructions may be loaded onto a computer or other 
programmable data processing apparatus to cause a series of operational steps to be 

20 performed on the computer or other programmable apparatus to produce a computer 
implemented process such that the instructions which execute on the computer or 
other programmable apparatus provide steps for implementing the functions specified 
in the diagram block or blocks. 

Although the present disclosure describes the processing of local and remote 

25 audio information, it will be understood that embodiments according to the invention 
can be practiced utilizing video information that accompanies the local and remote 
audio information. Accordingly, as the audio information is time-expanded and/or 
compressed, the accompanying video information may also be processed to, for 
example, maintain synchronization between the video information and the 

30 accompanying audio information. The synchronization of audio and video 

information is well known in the art and is described, for example, in US Patent No. 
6,249,319, by Post, entitled Method and apparatus for finding a correct 
synchronization point within a data stream, which is commonly assigned to the 
present assignee, the disclosure of which is incorporated herein in its entirety. 
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Embodiments according to the invention can provide methods, systems, and 
computer program products for changing a time interval in which audio information 
occurs to provide time-changed audio information that can be combined with other 
audio information for broadcast. For example, in some embodiments according to the 
5 invention, local audio information can be broadcast over a satellite link to a remote 
site. The local audio information can be time-expanded so that the time interval 
during which the time-expanded audio occurs occupies some of the interval which 
would otherwise be silent due to the propagation delay of response from the remote 
site. When the remote audio information is received at the local site, the remote audio 

10 information can be combined with the time-expanded audio to provide audio 

information for broadcast to an audience. Therefore, embodiments according to the 
invention may allow an awkward silence that would otherwise exist in an interval 
between the end of the local audio and the beginning of the remote audio to be 
reduced to a comfortable level. 

15 Figure 1A is a block diagram that illustrates embodiments of methods, 

systems, and computer program products according to the invention. In particular, 
local audio information is provided by a local site 105 and transmitted to a remote site 
1 15 over a communications link 120, such as a satellite system 1 10. The local site 
105 can be, for example, a broadcast studio wherein a participant, such as an 

20 interviewer, can pose questions to another participant located at the remote site 115. 

The local audio information propagates to the remote site 1 15 via an uplink 
1 1 1 A and a downlink 1 1 IB associated with the satellite system 1 10. The local audio 
information provided by the local site 105 can be provided to a participant located at 
the remote site 1 15 who can, in turn, respond to a question or information provided by 

25 the local audio information. 

The uplink 111A and the downlink 11 IB each have respective propagation 
delays associated therewith. In other words, the time needed for audio information to 
propagate from the local site 105 to the remote site 1 15 includes a propagation delay 
for the uplink 1 1 1 A and a propagation delay for the downlink 1 1 IB, the total of which 

30 can be expressed by At]. It will be understood that a normal delay can exist between 
the time that the local audio information arrives at the remote site 115 and the time at 
which a response (in the form of remote audio information) is transmitted back to the 
local site 105, which can be expressed as At2. 
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In turn, remote audio information is transmitted from the remote site 1 15 to 
the local site 105 over the satellite system 110. The remote audio information is 
provided to the local site 105 over an uplink 1 12A and a downlink 1 12B, each of 
which has an associated propagation delay, the total of which can be expressed as At 3 . 
5 Accordingly, the "round trip" propagation delay associated with the local audio 

information and the response thereto, in the form of the remote audio information, can 
be expressed as: 

Ati + At 2 + At 3 . 

10 

It will be understood that the components of the propagation delay included in Atj and 
At3 can, if unprocessed, introduce awkward silence into the interaction between 
participants at the local site 105 and the remote site 115. 

According to embodiments of the invention, the local audio information 

15 transmitted to the remote site 1 15 is also provided to a processor circuit 125 which 
can change the time interval over which the local audio information normally occurs. 
For example, in some embodiments according to the invention, the processor circuit 
125 expands the time interval in which the local audio information occurs such that a 
time-expanded audio information occupies at least a portion of an awkward silence 

20 which may otherwise exist between unprocessed local audio information and the 

remote audio information. The time-expansion of audio information is well known in 
the art and is described, for example, in US Patent No. 6,353,632, to Moeller et al., 
entitled Video processing system with real time program duration compression and 
expansion, the disclosure of which is incorporated herein in its entirety. 

25 The time-expanded audio information is combined with the remote audio 

information by a combiner circuit 130 to provide broadcast audio which can be 
provided to an audience. Accordingly, from the perspective of the broadcast 
audience, the awkward silence may be reduced or eliminated, thereby adjusting the 
interaction between the local participant and the remote participant to have a more 

30 natural appearance to the broadcast audience. 

Figures IB and 1C are schematic timing diagrams that illustrate differences 
between unprocessed audio information, which may include awkward periods of 
silence, and processed audio information according to the invention. In particular, 
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Figure IB illustrates local audio information provided to the remote site 115. The 
propagation delay associated with the transmission of the local audio information to 
the remote site 1 15 is represented by Ati in Figure IB. The time internal At 2 
represents the time taken by the participant at remote site 1 15 to provide a response to 
5 the local audio information in the form of the remote information transmitted back to 
the local site 105. The propagation delay associated with the remote audio 
information is represented by the time interval At 3 in Figure IB. 

Accordingly, the total time interval observed by the participant at the local site 
105 from the transmission of the local audio information to the reception of the 

10 remote audio information at the local site 105 is represented by At] + At 2 + At3. As 
discussed above, it will be understood that the time interval At 2 can represent a 
normal interval of silence that is expected to occur between a question or request to a 
participant and the response provided thereto. 

As shown in Figure 1C, broadcast audio information includes time-expanded 

15 audio information that is expanded to occupy at least a portion of the time interval 

associated with the propagation delay of the local audio information to the remote site 
115 and the propagation delay associated with the transmission of the remote audio 
information from the remote site 115 to the local site 105. In particular, the time- 
expanded audio information is shown as occurring during a time interval that is 

20 greater than the interval in which the unprocessed local audio information occurs in 
Figure IB, which is expressed as Ati + At 3 in Figure 1C. Furthermore, as shown in 
Figure 1C, the expected silent time interval represented by At 2 is preserved in the 
broadcast audio such that the broadcast audience may perceive a normal period of 
delay in the interaction between the participants that may otherwise not exist in the 

25 unprocessed audio information. 

The broadcast audio further includes the remote audio information that is 
provided by the participant at the remote site 115. It will also be understood that 
although the term "silence" is used herein to describe a time interval (or delay) 
between a question or request and a response thereto. It will be understood that this 

30 time interval may include other audio information which is not responsive to a 

question or request included in the local audio information. For example, the time 
interval can include background noise or other ambient sounds which may be 
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introduced into the broadcast audio from another source or provided by the remote 
site 1 15 or the local site 105 as part of the remote or local audio information. 

Figure 2 is a flowchart that illustrates the operations of embodiments of 
methods, systems, and computer program products according to the invention. In 
5 particular, the local site generates local audio information (block 205) which is time 
expanded based on a delay associated with the propagation of the local audio 
information the remote audio information (block 210). For example, the propagation 
delay associated with the transmission of the local and remote audio information can 
be based on actual measurements or estimates of the propagation delay imposed by 

10 the communications link over which the local and remote audio information are 
transmitted. Furthermore, it will be understood that the delay associated with the 
propagation of local and remote audio information can be initially estimated and 
otherwise measured on an ongoing basis as the audio information is actually 
transmitted and received. Accordingly, the delay on which the time-expansion is 

15 based can change as conditions associated with the communications link change. For 
example, the satellite position may change over time thereby introducing more or less 
propagation delay which can cause a corresponding increase or decrease in the time 
interval over which the local audio information is expanded. 

The local audio information is transmitted to the remote site via the 

20 communications link (block 215). After a propagation delay Ati, the local audio 

information is received at the remote site. A participant at the remote site can provide 
a response to the received local audio information as the remote audio information. 
The remote audio information is transmitted from the remote site to the local site via 
the communications link (block 220) thereby experiencing a corresponding 

25 propagation delay At 3 . 

The time-expanded local audio information (Block 210) is combined with the 
remote audio information received at the local site to provide combined audio 
information (block 225). The combined audio information is broadcast to an audience 
which may not include participants at the local and remote sites (Block 230). 

30 It will be understood that the local audio information and the remote audio 

information may be transmitted by over communications links each having an 
associated propagation delay and/or basis for estimating the delay associate therewith. 
For example, the local audio information may be transmitted to the remote site over a 
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land line whereas the remote audio information can be transmitted to the local site via 
a wireless communications link such as the satellite link described above. It will be 
understood that other types of wireless links having associated propagation delays 
associated therewith can also be used. Moreover, different types of land lines having 
5 different propagation delays may also be used. 

Figure 3A is a block diagram that illustrates embodiments of methods, 
systems, and computer program products according to the invention. In particular, a 
local site 305 produces local audio information which is transmitted to a remote site 
315 over a communications link 320 provided by a satellite communication system 
10 310. It will be understood that even though a communications link 320 is shown as 
including an uplink and downlink portion, it will be understood that the 
communications link can include other components such as land line segments and 
other types of systems that may cause propagation delay in addition to the uplink and 
downlink portions. 

15 As the local audio information is provided by the local site 305, the local 

audio information is time-expanded, by a processor circuit 325, to occupy a time 
interval that is greater than a time interval in which the local audio information would 
otherwise occur to provide time-expanded audio information as the local audio 
information is generated. 

20 As discussed above, the local audio information is transmitted to the remote 

site 315, which provides the remote audio information to the local site 305 after the 
delay Atj + At 2 + At3. The local site 305 provides the received remote audio 
information to a processor circuit 335 that time-compresses the remote audio 
information to occur in a time interval that is less than a time interval in which the 

25 remote audio information would otherwise occur. The processor circuit 335 operates 
to provide time-compressed remote audio information responsive to a determination 
that the local audio and the remote information occur during an overlapping time 
interval. The time-compressed remote audio information and the time-expanded local 
audio information are combined by a combiner circuit 330 to produce broadcast 

30 audio. 

Figure 3B is a schematic timing diagram that illustrates local and remote audio 
information that occur during an overlapping time interval. In particular, according to 
Figure 3B, local audio information is provided by the local site 305 to the remote site 
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315. In turn, the remote audio information is provided to the local site 305. In some 
cases, the participants can begin talking "over" one another such that the local and 
remote audio information occur during the overlapping time interval, such as when a 
local participant and a remote participant become confused because of the 
5 propagation delay associated with the communications link 320. 

Figure 3C is a schematic timing diagram that illustrates broadcast audio 
including time-expanded local audio information combined with time-compressed 
remote audio information to provide the broadcast audio information. In particular, 
time-expanded audio information is provided by the time expansion circuit 325 until a 

10 time Ati wherein it is determined that remote audio information is also being received 
at the same time. Accordingly, the processor circuit 325 ceases the time-expansion of 
the local audio information, whereas the processor circuit 335 begins providing time- 
compressed remote audio information so as to introduce a time interval At 4 between 
the time-expanded local audio information and the time-compressed remote audio 

15 information. The time compressed remote audio information and the time expanded 
local audio information are combined to provide the broadcast audio which can 
provide the broadcast audience with the perception that a natural delay exists between 
the local audio information and the remote audio information. 

Figure 4 is a flowchart that illustrates operations of embodiments of methods, 

20 systems, and computer program products according to the invention. In particular, the 
local site generates local audio information which is time expanded to occur in a time 
interval that is based on a delay associated with the propagation local and remote 
audio information (block 405). The local audio information is transmitted from the 
local site to the remote site (block 415). The remote site transmits remote audio 

25 information to the local site 305 after a time interval At 2 , which can be a delay that is 
normally observed between a question and an answer thereto. The remote audio 
information is received at the local site 305 after a propagation delay At 3 (block 420). 

A determination is made that the local and remote audio information occur 
during an overlapping time interval (block 425). Such an occurrence can be a 

30 situation where participants at the local site 305 and at the remote site 350 appear to 
be talking over one another which can be due in part to the unusually long time delay 
between a question and answer introduced by the propagation delays At] and At3. 
Upon determining that the local audio information and the remote audio information 
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are occurring during the overlapping time interval, time-expansion of the local audio 
information (block 410) is ceased, whereas time-compression of the remote audio 
information begins (block 430). 

The time-compression of the remote audio information coupled with the 
5 ceased time-expansion of the local audio information can provide a delay between the 
local and remote audio information that is expected by a broadcast audience. The 
local audio information (both the time-expanded local audio information and the 
unexpanded local audio information) are combined with time-compressed remote 
audio information to provide the broadcast audio (block 435). The combined audio 

10 information can be broadcast to an audience (block 440) which can reduce the time 
period which participants at local and remote sites would otherwise perceive as 
talking over one another. 

Figure 5 is a block diagram that illustrates embodiments of methods, systems, 
and computer program products according to the invention. In particular, a local site 

15 505 and a remote site 515 are connected via a network 500. Participants at the local 
site 505 and at the remote site 515 can communicate with one another, such as during 
a conference call. It will be understood that other participants can function as the 
broadcast audience. 

As disclosed above, local audio information is transmitted from the local site 

20 505 to the remote site 515 experiencing a propagation delay Atj. Remote audio 

information is transmitted from the remote site 515 to the local site 505 experiencing 
a propagation delay At 3 . It will be understood that the remote audio information can 
be transmitted responsive to the reception of the local audio information after a delay 
which is normally expected to occur between reception of the local audio information 

25 and the transmission of the remote audio information. The local audio information 

and the remote audio information can be transmitted using a protocol known as Voice 
Over IP (VOIP). VOIP is well known in the art and is described, for example, in US 
Patent No. 6,421,424, entitled, Client simulator and method of operation for testing 
PSTN-to-IP network telephone services for individual & group internet clients prior 

30 to availability of the services, which is commonly assigned to the present assignee, 
the disclosure of which is incorporated herein in its entirety. 

Accordingly, the respective propagation delays associated with the local 
audio information and remote audio information can be different from one another 
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and, moreover, can change over time. Therefore, the propagation delay Ati associated 
with the local audio information can be different than the propagation delay At 3 
associated with the remote audio information at a first time, whereas later the 
propagation delays may be equal to one another. Accordingly, the delay used as the 
5 basis for the time-expansion or time-compression may change dynamically as 

communication occurs between the local site 505 and the remote site 515. It will be 
understood that embodiments according to the invention can be utilized, for example, 
in telephone handsets that may be used to provide conference calling functions. 
Alternatively, embodiments according to the invention can be utilized, for example, at 

10 least partially as part of a communications network that users may access to utilize the 
conference call features. 

It will be understood that the propagation delays associated with the local and 
remote audio information can be provided by conventional techniques used to provide 
a certain quality of service to users of the network 500. Quality of service techniques 

15 are well known in the art and are described, for example, in US Patent No. 5,694,548, 
entitled, System and method for providing multimedia quality of service sessions in a 
communications network, which is commonly assigned to the present assignee, the 
disclosure of which is incorporated herein in its entirety. Quality of service is not 
discussed further herein. 

20 It will also be understood that the other participants (described above as 

members of the broadcast audience) may join the conversation with the local and 
remote sites 505, 515, and thereby become participants. Accordingly, it will be 
understood that the definition of which of the participants represents the local site 505 
and the remote site 515 may change. For example, initially, the local site 505 and the 

25 remote site 515 may be the sole participants in a conference call. Later, a third site 
550 may begin communicating with the local and remote sites 505, 515. 

Moreover, audio information communicated to and from the third site 550, 
having respective propagation delays associated therewith, can also be processed 
according to embodiments of the invention. For example, if the propagation delays 

30 associated with the third site 550 are determined to introduce an awkward delay into 
the communication, the associated audio information can be processed to time-expand 
the respective local audio information provided by the third site 550 and/or time- 
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compress the remote audio information provided to the third site 550 as described 
above in detail. 

As discussed above, embodiments according to the invention can provide 
methods, systems, and computer program products for changing a time interval in 
5 which audio information occurs to provide time-changed audio information that can 
be combined with other audio information for broadcast. For example, in some 
embodiments according to the invention, local audio information can be broadcast 
over a satellite link to a remote site. The local audio information can be time- 
expanded so that the time interval during which the time-expanded audio occurs 

10 occupies some of the interval which would otherwise be silent due to the propagation 
delay of response from the remote site. When the remote audio information is 
received at the local site, the remote audio information can be combined with the 
time -expanded audio to provide audio information for broadcast to an audience. 
Therefore, embodiments according to the invention may allow an awkward silence 

15 that would otherwise exist in an interval between the end of the local audio and the 
beginning of the remote audio to be reduced to a comfortable level. 

Many alterations and modifications may be made by those having ordinary 
skill in the art, given the benefit of present disclosure, without departing from the 
spirit and scope of the invention. Therefore, it must be understood that the illustrated 

20 embodiments have been set forth only for the purposes of example, and that it should 
not be taken as limiting the invention as defined by the following claims. The 
following claims are, therefore, to be read to include not only the combination of 
elements which are literally set forth but all equivalent elements for performing 
substantially the same function in substantially the same way to obtain substantially 

25 the same result. The claims are thus to be understood to include what is specifically 
illustrated and described above, what is conceptually equivalent, and also what 
incorporates the essential idea of the invention. 
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